Descent methods with linesearch in the presence of perturbations

AbstractWe consider the class of descent algorithms for unconstrained optimization with an Armijo-type stepsize rule in the case when the gradient of the objective function is computed inexactly. An important novel feature in our theoretical analysis is that perturbations associated with the gradient are not assumed to be relatively small or to tend to zero in the limit (as a practical matter, we expect them to be reasonably small, so that a meaningful approximate solution can be obtained). This feature makes our analysis applicable to various difficult problems encounted in practice. We propose a modified Armijo-type rule for computing the stepsize which guarantees that the algorithm obtains a reasonable approximate solution. Furthermore, if perturbations are small relative to the size of the gradient, then our algorithm retains all the standard convergence properties of descent methods

Motion of Inertial Observers Through Negative Energy

Recent research has indicated that negative energy fluxes due to quantum coherence effects obey uncertainty principle-type inequalities of the form |\Delta E|\,{\Delta \tau} \lprox 1\,. Here $|\Delta E|$ is the magnitude of the negative energy which is transmitted on a timescale $\Delta \tau$. Our main focus in this paper is on negative energy fluxes which are produced by the motion of observers through static negative energy regions. We find that although a quantum inequality appears to be satisfied for radially moving geodesic observers in two and four-dimensional black hole spacetimes, an observer orbiting close to a black hole will see a constant negative energy flux. In addition, we show that inertial observers moving slowly through the Casimir vacuum can achieve arbitrarily large violations of the inequality. It seems likely that, in general, these types of negative energy fluxes are not constrained by inequalities on the magnitude and duration of the flux. We construct a model of a non-gravitational stress-energy detector, which is rapidly switched on and off, and discuss the strengths and weaknesses of such a detector.Comment: 18pp + 1 figure(not included, available on request), in LATEX, TUPT-93-

Low Complexity Regularization of Linear Inverse Problems

Inverse problems and regularization theory is a central theme in contemporary signal processing, where the goal is to reconstruct an unknown signal from partial indirect, and possibly noisy, measurements of it. A now standard method for recovering the unknown signal is to solve a convex optimization problem that enforces some prior knowledge about its structure. This has proved efficient in many problems routinely encountered in imaging sciences, statistics and machine learning. This chapter delivers a review of recent advances in the field where the regularization prior promotes solutions conforming to some notion of simplicity/low-complexity. These priors encompass as popular examples sparsity and group sparsity (to capture the compressibility of natural signals and images), total variation and analysis sparsity (to promote piecewise regularity), and low-rank (as natural extension of sparsity to matrix-valued data). Our aim is to provide a unified treatment of all these regularizations under a single umbrella, namely the theory of partial smoothness. This framework is very general and accommodates all low-complexity regularizers just mentioned, as well as many others. Partial smoothness turns out to be the canonical way to encode low-dimensional models that can be linear spaces or more general smooth manifolds. This review is intended to serve as a one stop shop toward the understanding of the theoretical properties of the so-regularized solutions. It covers a large spectrum including: (i) recovery guarantees and stability to noise, both in terms of $\ell^2$-stability and model (manifold) identification; (ii) sensitivity analysis to perturbations of the parameters involved (in particular the observations), with applications to unbiased risk estimation ; (iii) convergence properties of the forward-backward proximal splitting scheme, that is particularly well suited to solve the corresponding large-scale regularized optimization problem

Unruh--DeWitt detectors in spherically symmetric dynamical space-times

In the present paper, Unruh--DeWitt detectors are used in order to investigate the issue of temperature associated with a spherically symmetric dynamical space-times. Firstly, we review the semi-classical tunneling method, then we introduce the Unruh--DeWitt detector approach. We show that for the generic static black hole case and the FRW de Sitter case, making use of peculiar Kodama trajectories, semiclassical and quantum field theoretic techniques give the same standard and well known thermal interpretation, with an associated temperature, corrected by appropriate Tolman factors. For a FRW space-time interpolating de Sitter space with the Einstein--de Sitter universe (that is a more realistic situation in the frame of $\Lambda$CDM cosmologies), we show that the detector response splits into a de Sitter contribution plus a fluctuating term containing no trace of Boltzmann-like factors, but rather describing the way thermal equilibrium is reached in the late time limit. As a consequence, and unlike the case of black holes, the identification of the dynamical surface gravity of a cosmological trapping horizon as an effective temperature parameter seems lost, at least for our co-moving simplified detectors. The possibility remains that a detector performing a proper motion along a Kodama trajectory may register something more, in which case the horizon surface gravity would be associated more likely to vacuum correlations than to particle creation.Comment: 19 pages, to appear on IJTP. arXiv admin note: substantial text overlap with arXiv:1101.525

Maximal monotonicity, conjugation and the duality product

Recently, the authors studied the connection between each maximal monotone operator T and a family H(T) of convex functions. Each member of this family characterizes the operator and satisfies two particular inequalities. The aim of this paper is to establish the converse of the latter fact. Namely, that every convex function satisfying those two particular inequalities is associated to a unique maximal monotone operator.C

Forcing strong convergence of proximal point iterations in a Hilbert space

This paper concerns with convergence properties of the classical proximal point algorithm for finding zeroes of maximal monotone operators in an infinite-dimensional Hilbert space. It is well known that the proximal point algorithm converges weakly to a solution under very mild assumptions. However, it was shown by Güler [11] that the iterates may fail to converge strongly in the infinite-dimensional case. We propose a new proximal-type algorithm which does converge strongly, provided the problem has a solution. Moreover, our algorithm solves proximal point subproblems inexactly, with a constructive stopping criterion introduced in [31]. Strong convergence is forced by combining proximal point iterations with simple projection steps onto intersection of two halfspaces containing the solution set. Additional cost of this extra projection step is essentially negligible since it amounts, at most, to solving a linear system of two equations in two unknowns

On The Behaviour Of Constrained Optimization Methods When Lagrange Multipliers Do Not Exist

Sequential optimality conditions are related to stopping criteria for nonlinear programming algorithms. Local minimizers of continuous optimization problems satisfy these conditions without constraint qualifications. It is interesting to discover whether well-known optimization algorithms generate primal-dual sequences that allow one to detect that a sequential optimality condition holds. When this is the case, the algorithm stops with a correct diagnostic of success (convergence). Otherwise, closeness to a minimizer is not detected and the algorithm ignores that a satisfactory solution has been found. In this paper it will be shown that a straightforward version of the Newton-Lagrange (sequential quadratic programming) method fails to generate iterates for which a sequential optimality condition is satisfied. On the other hand, a Newtonian penalty-barrier Lagrangian method guarantees that the appropriate stopping criterion eventually holds. © 2013 © 2013 Taylor & Francis.293646657Andreani, R., Martínez, J.M., Schuverdt, M.L., On the relation between the constant positive linear dependence condition and quasinormality constraint qualification (2005) J. Optim. Theory Appl, 125, pp. 473-485. , doi: 10.1007/s10957-004-1861-9Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L., On augmented Lagrangian methods with general lower-level constraints (2007) SIAM J. Optim, 18, pp. 1286-1309. , doi: 10.1137/060654797Andreani, R., Martínez, J.M., Svaiter, B.F., A new sequential optimality condition for constrained optimization and algorithmic consequences (2010) SIAM J. Optim, 20, pp. 3533-3554. , doi: 10.1137/090777189Andreani, R., Haeser, G., Martínez, J.M., On sequential optimality conditions for smooth constrained optimization (2011) Optimization, 60, pp. 627-641. , doi: 10.1080/02331930903578700Andreani, R., Haeser, G., Schuverdt, M.L., Silva, P.J.S., Two new weak constraint qualifications and applications (2012) SIAM J. Optim, 22, pp. 1109-1135. , doi: 10.1137/110843939Andreani, R., Haeser, G., Schuverdt, M.L., Silva, P.J.S., A relaxed constant positive linear dependence constraint qualification and applications (2012) Math. Program, 135, pp. 255-273. , doi: 10.1007/s10107-011-0456-0Arutyunov, A.V., (2000) Optimality Conditions - Abnormal and Degenerate Problems, , Kluwer, DordrechtBartholomew-Biggs, M.C., Recursive quadratic programming methods based on the augmented Lagrangian (1987) Math. Program. Study, 31, pp. 21-41. , doi: 10.1007/BFb0121177Benson, H.Y., Shanno, D.F., Vanderbei, R.J., Interior-point methods for nonconvex nonlinear programming - Filter methods and merit functions (2002) Comput. Optim. Appl, 23, pp. 257-272. , doi: 10.1023/A:1020533003783Bertsekas, D.P., (1999) Nonlinear Programming, , Athena Scientific, Belmont, MABielschowsky, R.H., Gomes, F.A.M., Dynamic control of infeasibility in equality constrained optimization (2008) SIAM J. Optim, 19, pp. 1299-1325. , doi: 10.1137/070679557Byrd, R.H., Gilbert, J.Ch., Nocedal, J., A trust region method based on interior point techniques for nonlinear programming (2000) Math. Program, 89, pp. 149-185. , doi: 10.1007/PL00011391Byrd, R.H., Nocedal, J., Waltz, R.A., KNITRO - An integrated package for nonlinear optimization (2006) Large-Scale Nonlinear Optimization, pp. 35-59. , in, G. Di Pillo and M. Roma, eds. Springer, New YorkConn, A.R., Gould, N.I.M., Toint, Ph.L., (2000) Trust Region Methods, , MPS/SIAM Series on Optimization, SIAM, Philadelphia, PAContesse-Becker, L., Extended convergence results for the method of multipliers for non-strictly binding inequality constraints (1993) J. Optim. Theory Appl, 79, pp. 273-310. , doi: 10.1007/BF00940582Di Pillo, G., Lucidi, S., An augmented Lagrangian function with improved exactness properties (2001) SIAM J. Optim, 12, pp. 376-406. , doi: 10.1137/S1052623497321894Di Pillo, G., Liuzzi, G., Lucidi, S., Palagi, L., An exact augmented Lagrangian function for nonlinear programming with two-sided constraints (2003) Comput. Optim. Appl, 25, pp. 57-83. , doi: 10.1023/A:1022948903451Di Pillo, G., Liuzzi, G., Lucidi, S., Palagi, L., A truncated Newton method in an augmented Lagrangian framework for nonlinear programming (2010) Comput. Optim. Appl, 45, pp. 311-352. , doi: 10.1007/s10589-008-9216-3Di Pillo, G., Liuzzi, G., Lucidi, S., An exact penalty-Lagrangian approach for large-scale nonlinear programming (2011) Optimization, 60, pp. 223-252. , doi: 10.1080/02331934.2010.505964Fan, J.Y., Yuan, Y.X., On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption (2005) Computing, 34, pp. 23-39. , doi: 10.1007/s00607-004-0083-1Fernández, D., A quasi-Newton strategy for the sSQP method for variational inequality and optimization problems (2011) Math. Program, pp. 199-223. , doi: 10.1007/s10107-011-0493-8Fernández, D., Solodov, M., Stabilized sequential quadratic programming for optimization and a stabilized Newton-Type method for variational problems (2010) Math. Program, 125, pp. 47-73. , doi: 10.1007/s10107-008-0255-4Fletcher, R., (1987) Practical Methods of Optimization, , Academic Press, LondonFletcher, R., Leyffer, S., Toint, Ph.L., On the global convergence of a filter-SQP algorithm (2002) SIAM J. Optim, 13, pp. 44-59. , doi: 10.1137/S105262340038081XFletcher, R., Gould, N.I.M., Leyffer, S., Toint, Ph.L., Wächter, A., Global convergence of trust-region SQP-filter algorithms for general nonlinear programming (2002) SIAM J. Optim, 13, pp. 635-659. , doi: 10.1137/S1052623499357258Giannessi, F., (2005) Separation of Sets and Optimality Conditions, , Springer, New YorkGould, N.I.M., Toint, Ph.L., Nonlinear programming without a penalty function or a filter (2010) Math. Program, 122, pp. 155-196. , doi: 10.1007/s10107-008-0244-7Gratton, S., Mouffe, M., Toint, Ph.L., Stopping rules and backward error analysis for bound-constrained optimization (2011) Numer. Math, 119, pp. 163-187. , doi: 10.1007/s00211-011-0376-1Izmailov, A., Solodov, M., Stabilized SQP revisited (2010) Math. Program, pp. 93-120. , doi: 10.1007/s10107-010-0413-3Liu, X.W., Yuan, Y.X., A null-space primal-dual interior-point algorithm for nonlinear optimization with nice convergence properties (2010) Math. Program, 125, pp. 163-193. , doi: 10.1007/s10107-009-0272-yLiu, X.W., Yuan, Y.X., A sequential quadratic programming method without a penalty function or a filter for nonlinear equality constrained optimization (2011) SIAM J. Optim, 21, pp. 545-571. , doi: 10.1137/080739884Luksan, L., Matonoha, C., Vlcek, J., Interior point methods for large-scale nonlinear programming (2003) Optim. Methods Softw, 20, pp. 569-582. , doi: 10.1080/10556780500140508Luksan, L., Matonoha, C., Vlcek, J., Algorithm 896: LSA: Algorithms for large-scale optimization (2009) ACM Trans. Math. Softw, 36, pp. 161-1629. , doi: 10.1145/1527286.1527290Martínez, J.M., Santos, L.T., Some new theoretical results on recursive quadratic programming algorithms (1998) J. Optim. Theory Appl, 97, pp. 435-454. , doi: 10.1023/A:1022686919295Martínez, J.M., Svaiter, B.F., A practical optimality condition without constraint qualifications for nonlinear programming (2003) J. Optim. Theory Appl, 118, pp. 117-133. , doi: 10.1023/A:1024791525441Nocedal, J., Wright, S.J., (1999) Numerical Optimization, , Springer, New YorkQi, L., Wei, Z., On the constant positive linear dependence condition and its application to SQP methods (2000) SIAM J. Optim, 10, pp. 963-981. , doi: 10.1137/S1052623497326629Schiela, A., Guenther, A., An interior point algorithm with inexact step computation in function space for state constrained optimal control (2011) Numer. Math, 119, pp. 373-407. , doi: 10.1007/s00211-011-0381-4Shen, C., Xue, W., Pu, D., A filter SQP algorithm without a feasibility restoration phase (2009) Comput. Appl. Math, 28, pp. 167-194. , doi: 10.1590/S1807-03022009000200003Shen, C., Leyffer, S., Fletcher, R., Nonmonotone filter method for nonlinear optimization (2012) Comput. Optim. Appl, 52, pp. 583-607. , doi: 10.1007/s10589-011-9430-2Wächter, A., Biegler, L.T., On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming (2006) Math. Program, 106, pp. 25-57. , doi: 10.1007/s10107-004-0559-yWright, S.J., Superlinear convergence of a stabilized SQP method to a degenerate solution (1998) Comput. Optim. Appl, 11, pp. 253-275. , doi: 10.1023/A:1018665102534Wright, S.J., Modifying SQP for degenerate problems (2002) SIAM J. Optim, 13, pp. 470-497. , doi: 10.1137/S105262349833373